Skip to content

Add PiecewiseSequence take microbenchmarks#8830

Draft
danking wants to merge 10 commits into
codex/piecewise-sequence-cursor-copyfrom
codex/piecewise-sequence-benchmarks
Draft

Add PiecewiseSequence take microbenchmarks#8830
danking wants to merge 10 commits into
codex/piecewise-sequence-cursor-copyfrom
codex/piecewise-sequence-benchmarks

Conversation

@danking

@danking danking commented Jul 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

@robert3005 I'm not sure if you want to check these in. These are what I used to reason about various ways to copy slices of data from one spot in memory to another.

Summary

  • add piecewise_sequence_take_primitive to compare constant-length PiecewiseSequence take against dense lengths and reifying constant lengths
  • add take_slices_to_buffer_matrix to compare extend/cursor-copy loops with and without preverification plus unchecked slicing

Validation

  • cargo +nightly fmt --all
  • cargo clippy -p vortex-array --all-targets --all-features -- -D warnings
  • cargo bench -p vortex-array --bench piecewise_sequence_take_primitive
  • cargo bench -p vortex-array --bench take_slices_to_buffer_matrix

@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch from 18df62f to 04a21ec Compare July 17, 2026 19:12
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from 9adf56d to c4a0358 Compare July 17, 2026 19:40
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch from 0906157 to f5c8555 Compare July 17, 2026 19:40
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch 2 times, most recently from 80f7b89 to 30870aa Compare July 17, 2026 20:19
@danking danking added the changelog/chore A trivial change label Jul 17, 2026
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from 30870aa to e777ab4 Compare July 17, 2026 20:25
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch from f5c8555 to b59c45f Compare July 17, 2026 20:33
@danking
danking requested a review from robert3005 July 17, 2026 20:37
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from e777ab4 to e4a0543 Compare July 17, 2026 20:37
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch from b59c45f to 388d234 Compare July 17, 2026 20:41
@danking danking mentioned this pull request Jul 17, 2026
Draft
@robert3005

robert3005 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

I think we want benchmarks. I came up with the long term solution in #8825 and we have to make sure we don't regress

@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from 490e9db to ed7bbe2 Compare July 20, 2026 16:11
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch from 388d234 to 5f0d64e Compare July 20, 2026 16:11
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from ed7bbe2 to efafe71 Compare July 20, 2026 16:37
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch from 5f0d64e to 245abc0 Compare July 20, 2026 16:37
@codspeed-hq

codspeed-hq Bot commented Jul 20, 2026 •
edited
Loading

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 1798 untouched benchmarks
🆕 42 new benchmarks
⏩ 44 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation optimized_constant_lengths[1] N/A 47.5 ms N/A
🆕 Simulation optimized_constant_lengths[1024] N/A 7.9 ms N/A
🆕 Simulation optimized_constant_lengths[16] N/A 9.6 ms N/A
🆕 Simulation optimized_constant_lengths[256] N/A 8 ms N/A
🆕 Simulation optimized_constant_lengths[4] N/A 15 ms N/A
🆕 Simulation optimized_constant_lengths[64] N/A 8.2 ms N/A
🆕 Simulation prebuilt_dense_lengths[1] N/A 67.3 ms N/A
🆕 Simulation prebuilt_dense_lengths[1024] N/A 7.9 ms N/A
🆕 Simulation prebuilt_dense_lengths[16] N/A 10.9 ms N/A
🆕 Simulation prebuilt_dense_lengths[256] N/A 8 ms N/A
🆕 Simulation prebuilt_dense_lengths[4] N/A 19.9 ms N/A
🆕 Simulation prebuilt_dense_lengths[64] N/A 8.5 ms N/A
🆕 Simulation reify_constant_lengths_then_take[1] N/A 71.3 ms N/A
🆕 Simulation reify_constant_lengths_then_take[1024] N/A 7.9 ms N/A
🆕 Simulation reify_constant_lengths_then_take[16] N/A 11 ms N/A
🆕 Simulation reify_constant_lengths_then_take[256] N/A 8.1 ms N/A
🆕 Simulation reify_constant_lengths_then_take[4] N/A 20.7 ms N/A
🆕 Simulation reify_constant_lengths_then_take[64] N/A 8.6 ms N/A
🆕 Simulation cursor_copy_safe[1000] N/A 6.8 µs N/A
🆕 Simulation cursor_copy_safe[10000] N/A 14.2 µs N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Comparing codex/piecewise-sequence-benchmarks (a42849f) with codex/piecewise-sequence-cursor-copy (9183265)2

Open in CodSpeed

Footnotes

  1. 44 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on codex/piecewise-sequence-cursor-copy (182567a) during the generation of this report, so 1d362cc was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from efafe71 to 56b0394 Compare July 20, 2026 16:56
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch from 245abc0 to baaab82 Compare July 20, 2026 16:56
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from 56b0394 to 80bfa43 Compare July 20, 2026 18:10
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch 2 times, most recently from 0121842 to 4600066 Compare July 20, 2026 19:24
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from 80bfa43 to 09ca199 Compare July 20, 2026 19:24
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch 6 times, most recently from 431cbd5 to 4bb6ce4 Compare July 20, 2026 20:29
@danking
danking changed the base branch from codex/piecewise-sequential-run-take-consumers to codex/piecewise-sequence-cursor-copy July 20, 2026 20:30
danking added 10 commits July 20, 2026 18:02
@danking
Specialize VarBin PiecewiseSequence takes
080216e 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Use Columnar lengths in VarBin takes
fe7dd28 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Rely on PiecewiseSequence validation in VarBin takes
5d31540 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Use cursor copies for PiecewiseSequence slices
64231f7 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Advance PiecewiseSequence copy destinations
82bdd51 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Move spare buffer writer to vortex-buffer
182567a 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Add PiecewiseSequence take microbenchmarks
7464107 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Make take slices matrix cursor copy checks explicit
0435161 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Benchmark advancing PiecewiseSequence copies
78ad614 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
Use shared spare buffer writer in benchmarks
a42849f 
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
danking force-pushed the codex/piecewise-sequence-benchmarks branch from 4bb6ce4 to a42849f Compare July 20, 2026 22:04
@danking
danking force-pushed the codex/piecewise-sequence-cursor-copy branch from 9183265 to 182567a Compare July 20, 2026 22:04
@robert3005
robert3005 force-pushed the codex/piecewise-sequence-cursor-copy branch from 87f6283 to 8235e03 Compare July 21, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@robert3005 robert3005 Awaiting requested review from robert3005

Assignees

No one assigned

Labels

changelog/chore A trivial change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@danking @robert3005